Today many organizations concentrate their IT (information technology) spending on methods and technologies that help them reduce cost by increasing the efficiency and effectiveness of their IT infrastructure. A key pain point faced by many organization and companies is that, in the beginning of the Internet boom. also called the ‘dot-corn’ bubble, many organizations had to embrace the Internet overnight, and in doing so established infrastructure elements in an ad-hoc fashion that were not well-designed for scalability or growth. Today, these organizations and companies are faced with infrastructures that are loosely assembled, are costly to maintain, and are difficult and expensive to grow with the business needs. This is evident in both the business processes as well as in the way these organizations manage data.
In many instances, the data is fragmented within the organization or company, with different database systems being utilized in different departments, all of which often maintain essentially the same data in multiple formats using different database table designs. Here, it would be hugely beneficial to maintain all data a (logically) single place using a standardized schema. Having such centralized data warehouse or database engine would enable quick data analysis for improved customer relationship management, simplify development of new products, and reduce maintenance cost for the IT infrastructure itself while improving the reliability and availability of the entire system.
The international publication WO 99/52047A1 relates to a method and system for migrating data from one or more ASCII files and/or from one or more relational databases to one or more relational database tables without the need to write code. This allows the user to define mapping templates and conditionals to assist in translating and transforming data values. The method also enforces referential integrity, data dependencies, order of operations, and uniqueness constraints using a predefined set of migration rules templates that are based on the principles of relational design. The method uses these mapping and migration rules templates to generate instructions for updating or populating relational database destination tables. The instructions control the data transfer, data translation, data transformation, data validation, foreign key insertion, and the addition of required codes and flags in the destination tables. A migration engine of the system includes a data map architect and an update processor which spawns the templates and migrates the data dynamically, utilizing the data definitions for the destination tables.
This prior art has the drawback that it is limited to a particular type of database systems, in which Oracle® Application tables are implemented. This requires users to manually define rules for the migration or at least interactively, i.e. by user interaction. (Oracle is a trademark of Oracle Corporation). This prior art does not show provisions which allow to discover databases across an organization or augment a consolidation process with actual access patterns.
From the above follows that there is still a need in the art for an efficient scheme that allows to consolidate data from distributed databases into a central database.